Mixed-reality (MR) systems/devices include virtual-reality (VR) and augmented-reality (AR) systems. Conventional VR systems create completely immersive experiences by restricting users' views to only virtual images rendered in VR scenes/environments. Conventional AR systems create AR experiences by visually presenting virtual images that are placed in or that interact with the real world. As used herein, VR and AR systems are described and referenced interchangeably via use of the phrase “MR system.” As also used herein, the terms “virtual image” or “hologram” refer to any type of digital image rendered by an MR system. Furthermore, it should be noted that a head-mounted device (HMD) typically provides the display used by the user to view and/or interact with holograms provided within an MR scene.
A MR system's HMD typically includes a head tracking system having one or more head tracking camera(s) and inertial measurement units (IMUs). Using these cameras, the head tracking system beneficially determines the HMD's position and pose relative to its surrounding environment. It should be noted that the HMD's position and pose are both relied upon by many existing MR systems when visually placing/rendering holograms into mixed-reality (MR) scenes. By continuously or frequently determining position and pose, MR systems can provide display corrections to its rendered holograms. These display corrections enable the MR systems to render realistic holograms because they enable the MR systems to dynamically respond to HMD's movements in real-time and to update the holograms in response to those movements.
Some MR systems additionally use an inertial measurement unit (IMU) to monitor the HMD's acceleration changes in order to determine the HMD's position and pose. While IMU data is helpful in determining position and pose, IMU data should not be used in isolation for head tracking, pose/position estimation, or hologram rendering/placement. The reason is because an IMU calculates position using a square function of time, and any errors in that calculation are proportional to the IMU's sampling time. Consequently, in some cases, the determined position/pose (as determined by the IMU) may be significantly skewed as a result of the errors being multiplied by the time squared.
Despite the potential for skewed information, IMU data can still be highly valuable when determining position and pose. As such, some MR systems use a combination of information from their head tracking cameras and their IMUs, with the data from each combined using a Kalman filter, in order to accurately track and validate position and pose of the HMD over time. That is, IMU data can be used to augment or supplement the head tracking camera data, thereby resulting in a more reliable determination of the HMD's position and pose. A more reliable determination of position and pose allows for improved hologram placement.
Unfortunately, if the HMD is positioned within a moving environment (e.g., an airplane, automobile, ship, elevator, etc.), significant discrepancies can arise between display positioning information provided by the head tracking system and display positioning information provided by the IMU. These discrepancies result in conflicts to the HMD's determined position and pose. Such conflicts can, in turn, cause the MR system to misplace holograms within the MR scenes based on bad data. This misplacement is particularly pronounced for holograms that are supposed to be rendered at fixed locations. As such, there is a significant need to improve how holograms are placed within an MR scene, especially when rendering MR scenes for an HMD located in a moving environment.
The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practiced.